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Abstract 

This  article  describes  a  methodology  for  programming  robots  known  as  pwhahilistic  robotics.  The  proba¬ 
bilistic  paradigm  pays  tribute  to  the  inherent  uncertainty  in  robot  perception,  relying  on  explicit  represen¬ 
tations  of  uncertainty  when  determining  what  to  do.  This  article  surveys  some  of  the  progress  in  the  field, 
using  in-depth  examples  to  illustrate  some  of  the  nuts  and  bolts  of  the  basic  approach.  Our  central  con¬ 
jecture  is  that  the  probabilistic  approach  to  robotics  scales  better  to  complex  real-world  applications  than 
approaches  that  ignore  a  robot’s  uncertainty. 
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1  Introduction 


Building  autonomous  robots  has  been  a  central  objective  of  research  in  artificial  intelligence.  Over  the 
past  decades,  researchers  in  AI  have  developed  a  range  of  methodologies  for  developing  robotic  software, 
ranging  from  model-based  to  purely  reactive  paradigms.  More  than  once,  the  discussion  on  what  the  right 
way  might  be  to  program  robots  has  been  accompanied  with  speculations  concerning  the  very  nature  of 
4  intelligence  per  se,  in  animals  and  people. 

One  of  these  approaches,  probabilistic  robotics,  has  led  to  fielded  systems  with  unprecedented  levels  of 
autonomy  and  robustness.  While  the  roots  of  this  approach  can  be  traced  back  to  the  early  1960s,  in  recent 
^  years  the  probabilistic  approach  has  become  the  dominant  paradigm  in  a  wide  an*ay  of  robotic  problems. 

Probabilistic  algorithms  have  been  at  the  core  of  a  series  of  fielded  autonomous  robots,  exhibiting  an  un¬ 
precedented  level  of  performance  and  robustness  in  the  real  world.  These  recent  successes  can  be  attributed 
to  at  least  two  developments:  the  availability  of  immense  computational  resources  even  on  low-end  PCs 
and,  more  importantly,  fundamental  progress  on  the  basic  algorithmic  and  theoretical  levels. 

So  what  exactly  is  the  probabilistic  approach  to  robotics?  At  its  core  is  the  idea  of  representing  infor¬ 
mation  through  probability  densities.  In  particular,  probabilistic  ideas  can  be  found  in  perception,  i.e.,  the 
way  sensor  data  is  processed,  and  action,  i.e.,  the  way  decisions  are  made; 

•  Probabilistic  perception.  Robots  are  inherently  uncertain  about  the  state  of  their  environments.  Un¬ 
certainty  arises  from  sensor  limitations,  noise,  and  the  fact  that  most  interesting  environments  are — to  a 
certain  degree — unpredictable.  When  “guessing”  a  quantity  from  sensor  data,  the  probabilistic  approach 
computes  a  probability  distribution  over  what  might  be  the  case  in  the  world,  instead  of  generating  a 
single  “best  guess”  only.  As  a  result,  a  probabilistic  robot  can  gracefully  recover  from  errors,  handle 
ambiguities,  and  integrate  sensor  data  in  a  consistent  way.  Moreover,  a  probabilistic  robot  knows  about 
its  own  ignorance — a.  key  prerequisite  of  truly  autonomous  robots. 

•  Probabilistic  control.  Autonomous  robots  must  act  in  the  face  of  uncertainty — a  direct  consequence 
of  their  inability  to  know  what  is  the  case.  When  making  decisions,  probabilistic  approaches  take  the 
robot’s  uncertainty  into  account.  Some  consider  only  the  robot’s  current  uncertainty,  others  anticipate 
future  uncertainty.  Instead  of  considering  the  most  likely  situations  only  (current  or  projected),  many 
probabilistic  approaches  strive  to  compute  a  decision-theoretic  optimum,  in  which  decision  are  based  on 
all  possible  contingencies. 

These  two  items  are  the  basic  characterization  of  the  probabilistic  approach  to  robotics. 

What  is  the  benefit  of  programming  robots  probabilistically?  Our  central  conjecture  is  nothing  less  than 
the  following:  A  robot  that  carries  a  notion  of  its  own  uncertainty  and  that  acts  accordingly,  will  do  better 
than  one  that  does  not.  In  particular,  probabilistic  approaches  are  typically  more  robust  in  the  face  of  sensor 
limitations,  sensor  noise,  and  environment  dynamics.  They  often  scale  much  better  to  complex  environ- 
^  ments,  where  the  ability  to  handle  uncertainty  is  of  even  greater  importance.  In  fact,  certain  probabilistic 

algorithms  are  currently  the  only  known  working  solutions  to  hard  robotic  estimation  problems,  such  as  the 
kidnapped  robot  problem  [28],  in  which  a  mobile  robot  must  recover  from  localization  failure;  or  the  prob- 
•  lem  of  building  accurate  maps  of  very  large  environments,  in  the  absence  of  a  global  positioning  device  such 

as  GPS.  Additionally,  probabilistic  algorithms  make  much  weaker  requirements  on  the  accuracy  of  models 
than  many  classical  planning  algorithms  do,  thereby  relieving  the  programmer  from  the  (unsurmountable) 
burden  to  come  up  with  accurate  models.  Viewed  probabilistically,  the  robot  learning  problem  is  a  long-term 
estimation  problem.  Thus,  probabilistic  algorithms  provide  a  sound  methodology  for  many  flavors  of  robot 
learning.  And  finally,  probabilistic  algorithms  are  broadly  applicable  to  virtually  every  problem  involving 
perception  and  action  in  the  real  world. 
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However,  these  advantages  come  at  a  price.  Traditionally,  the  two  most  frequently  cited  limitations  of 
probabilistic  algorithms  are  computational  inejficiency,  and  a  need  to  approximate.  Certainly,  there  is  some 
truth  to  these  criticisms.  Probabilistic  algorithms  are  inherently  less  efficient  than  non-probabilistic  ones, 
due  to  the  fact  that  they  consider  entire  probability  densities.  However,  this  carries  the  benefit  of  increased 
robustness.  The  need  to  approximate  arises  from  the  fact  that  most  robot  worlds  are  continuous.  Computing 
exact  posterior  distributions  is  typically  infeasible,  since  distributions  over  the  continuum  possess  infinitely 
many  dimensions.  Sometimes,  one  is  fortunate  in  that  the  uncertainty  can  approximated  tightly  with  a  * 

compact  parametric  model  (e.g.,  discrete  distributions  or  Gaussians);  in  other  cases,  such  approximations 
are  too  crude  and  more  complex  representations  most  be  employed. 

None  of  these  limitations,  however,  pose  serious  obstacles.  Recent  research  has  led  to  a  range  of  algo-  ^ 

rithms  that  are  computationally  efficient  and  also  highly  accurate.  To  illustrate  probabilistic  algorithms  in 
practice,  this  article  describes  three  such  algorithms  in  detail,  and  argues  that  the  probabilistic  paradigm  is 
unique  in  its  ability  to  solve  hard  robotics  problems  in  uncertain  and  complex  worlds. 


2  Mobile  Robot  Localization 

Let  us  take  a  first,  deeper  look  into  a  specific  probabilistic  algorithm,  which  solves  an  important  problem  in 
mobile  robotics,  namely  that  of  localization.  Localization  is  the  problem  of  finding  out  a  robot’s  coordinates 
relative  to  its  environment,  assuming  that  one  is  provided  with  a  map  of  the  environment.  Localization  is  a 
key  component  in  various  successful  mobile  robot  systems  (see  e.g.,  [4,  51]).  Occasionally  is  has  been  re¬ 
ferred  to  as  “  the  most  fundamental  problem  to  providing  a  mobile  robot  with  autonomous  capabilities”  [16]. 
Particularly  challenging  is  the  global  localization  problem,  where  the  robot  does  not  know  its  initial  position 
and  therefore  has  to  globally  localize  itself. 

Approached  probabilistically,  the  localization  problem  is  a  density  estimation  problem,  where  a  robot 
seeks  to  estimate  a  posterior  distribution  over  the  space  of  its  poses  conditioned  on  the  available  data.  The 
term  pose,  in  this  article,  refers  to  a  robot’s  a-y-coordinates  together  with  its  heading  direction  9.  Denoting 
the  robot’s  pose  at  time  t  by  Sf,  and  the  data  leading  up  to  time  t  by  the  posterior  is  conveniently 
written  as 


p{sf\do..j,m).  (*) 

Here  m  is  the  model  of  the  world  (e.g.,  a  map).  For  brevity,  we  will  denote  this  posterior  6f,(s<),  and  refer  to 
it  as  the  robot’s  belief  state  at  time  t. 

Sensor  data  typically  comes  in  two  flavors:  Data  that  characterizes  the  momentary  situation  (e.g.,  camera 
images,  laser  range  scans),  and  data  relating  to  change  of  the  situation  (e.g.,  motor  controls  or  odometer 
readings).  Referring  to  the  former  as  obseix’ations  and  the  latter  as  action  data,  let  us  without  loss  of 
generality  assume  that  both  types  of  data  arrives  in  an  alternating  sequence: 

do...i  —  OQ,aQ,Oi,ai, . .  .,ai-i,Ot.  (2) 

Here  Ot  denote  the  observation  and  at  denotes  the  action  data  item  collected  at  time  t. 

To  estimate  the  desired  posterior  p(.s<  | do...t  probabilistic  approaches  frequently  resort  to  a  Markov 
assumption,  which  states  that  the  past  is  independent  of  the  future  given  knowledge  of  the  current  state.  The 
Markov  assumption  is  often  referred  to  as  the  static  world  assumption,  since  it  assumes  the  robot’s  pose  is 
the  only  state  in  the  world  that  would  impact  more  than  just  one  isolated  sensor  reading.  Practical  experience 
suggests,  however,  that  probabilistic  algorithms  are  robust  to  mild  violations  of  the  Markov  assumption,  and 
extensions  exist  that  go  beyond  this  assumption  (e.g.,  [33]). 
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(a)  Uniform  distribution,  path  (b)  Belief  6  at  second  pose  (c)  Belief  6  at  third  pose  (d)  Belief  b  at  fourth  pose 


Figure  3:  Example  of  grid-based  Markov  localization  in  a  symmetric  environment,  (a)  robot  path,  highlighting  four  robot  poses, 
(b)  to  (d):  belief  at  2nd,  3rd,  and  4th  pose,  respectively. 


factors  of  the  belief 

2.  For  this  guess  a  successor  pose  .S/,  according  to  the  distribution  ???). 

3.  Assign  a  preliminary  importance  factor  7^(0/ |.Sf,  ???)  to  this  sample  and  add  it  to  the  new  sample  set  repre¬ 
senting  bf{si). 

4.  Repeat  Step  1  through  37?7  times.  Finally,  normalize  the  importance  factors  in  the  new  sample  set  bt{si) 
so  that  they  add  to  1 . 

Figure  4  shows  MCL  in  action.  Shown  in  the  first  diagram  is  a  belief  distribution  (sample  set)  at  the 
beginning  of  the  experiment  when  the  robot  does  not  (yet)  know  its  position.  Each  dot  is  a  three-dimensional 
sample  of  the  robot’s  ,r-?/-location  along  with  its  heading  direction  0.  The  second  diagram  shows  the  belief 
after  a  short  motion  segment,  incorporating  several  sonar  readings.  At  this  point,  most  samples  concentrate 
on  two  locations;  however,  the  symmetry  of  the  corridor  makes  it  impossible  to  disambiguate  them.  Finally, 
the  third  diagram  in  Figure  4  shows  the  belief  after  the  robot  moves  into  one  of  the  rooms,  enabling  it  to 
disambiguate  its  location  with  high  confidence. 

The  MCL  algorithm  is  in  fact  quite  efficient;  slight  modifications  of  the  basic  algorithms  [56,  92]  re¬ 
quire  as  few  as  100  samples  for  reliable  localization,  consuming  only  a  small  fraction  of  time  available 
on  low-end  PC.  It  can  also  be  implemented  as  any-time  algorithm  [17,  95],  meaning  that  it  can  adapt  to 
the  available  computational  resources  by  dynamically  adjusting  the  number  of  samples  m.  With  slight 
modifications — such  as  sampling  from  the  observation  [92] — MCL  has  been  shown  to  recover  gracefully 
from  global  localization  failures,  such  as  manifested  in  the  kidnapped  robot  problem  [27],  where  a  well- 
g  localized  robot  is  teleported  to  some  random  location  without  being  told.  For  these  reasons,  probabilistic 

algorithms  like  MCL  are  currently  the  best  known  methods  for  such  hard  localization  problems. 

Another  feature  of  MCL  is  that  its  models — in  particular  p{s\a^  s,  m),p{o\s,  m)  and  the  map — can  be 
•  extremely  crude  and  simplistic,  since  probabilistic  models  carry  their  own  notion  of  uncertainty.  This  makes 

probabilistic  algorithms  relatively  easy  to  code.  In  comparison,  traditional  robotics  algorithms  that  rely  on 
deterministic  models  make  much  stronger  demands  on  the  accuracy  of  the  underlying  models. 
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Figure  1:  Probabilistic  generalization  of  mobile  robot  kinematics:  Each  dark  line  illustrates  a  commanded  robot  path  and  the 
grayly  shaded  shows  the  posterior  distribution  of  the  robot’s  pose.  The  darker  an  area,  the  more  likely  it  is.  The  path  in  the  left 
diagram  is  40  meters  and  the  one  on  the  right  is  80  meters  long. 


The  desired  posterior  is  now  computed  using  a  recursive  formula,  which  is  obtained  by  applying  Bayes 
rule  and  the  theorem  of  total  probability  to  the  original  expression,  exploiting  the  Markov  assumption  twice: 

=  p{Sf\oo, . .  .,af^i,Of,rn) 

B^es 

-  Vt  P{ot\oo, . . . ,  Uf-i,  St.  m)  p{sf  |oo, . . . ,  a,_i,  m) 

—  Vt  Piot\sf,  m)  p(,Sf|oo, . .  m) 

Tot.^rob.  /I  \  f  y  t 

—  Vf  J  p(st\oo, ....  ni)  ,  at-i,  m) 

==  Vt  P{ot\^t,m)  J p{st\af_i^Sf^i.in)  p{sf^i\oo, ..  .,0^^1,711)  dsf^i 

—  Vt  p{Of\st.  717)  J  p{st\af^i,  Sf-i,  777)  dsf^i,  (3) 

Here  77,  is  a  constant  normalizer  which  ensures  that  the  result  sums  up  to  1.  Within  the  context  of  mobile 
robot  localization,  the  result  of  this  transformation 

btist)  =  mp(ot\st.m)  J p(st\at-i,st_i.m)bt-i{st_i)  dst-i  (4) 

is  often  referred  to  as  Markov  localization  [9,  32,  45,  49,  81,  87],  but  it  equally  represents  the  basic  up¬ 
date  equation  in  Kalman  filters  [48],  Hidden  Markov  models  [70],  and  dynamic  belief  networks  [18,  75] 
Kalman  filter  [48],  which  is  historically  the  most  popular  approach  for  position  tracking,  represents  beliefs 
by  Gaussians.  The  vanilla  Kalman  filter  also  assumes  Gaussian  noise  and  linear  motion  equations;  however, 
extensions  exist  that  relax  some  of  these  assumptions  [44,  63],  Kalman  filters  have  been  applied  with  great 
success  to  a  range  of  tracking  and  mapping  problems  in  robotics  [58,  83];  though  they  tend  not  to  work  well 
for  global  localization  or  the  kidnapped  robot  problem.  Markov  localization  using  discrete,  topological  rep¬ 
resentations  for  b  were  pioneered  (among  others)  by  Simmons  and  Koenig  [81],  whose  mobile  robot  Xavier 
traveled  more  than  230  kilometers  through  CMU’s  hallways  over  a  period  of  several  years  [80], 

To  implement  Equation  (4),  one  needs  to  specify  m)  andp(o,|st,  m).  Both  densities  are 

usually  time-invariant,  that  is,  they  do  not  depend  on  t,  hence  the  time  index  can  be  omitted.  The  first  density 
charactenzes  the  effect  of  the  robot’s  actions  a  on  its  pose,  and  can  therefore  be  viewed  as  a  probabilistic 
generalization  of  mobile  robot  kinematics;  see  Figure  1  for  examples.  The  other  density,  p(o\s,  m),  is  a 
probabilistic  model  of  perception.  Figure  2  illustrates  a  sensor  model  for  range  finders,  which  uses  ray¬ 
tracing  and  a  mixture  of  four  parametric  densities  to  calculate  p(o|s,  m).  In  our  implementation,  both  of 
these  probabilistic  models  are  quite  crude,  using  uncertainty  to  account  for  model  limitations  [32], 
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Figure  2:  Probabilistic  sensor  model  for  laser  range  finders:  (a)  The  density  p(o|s,  m)  relates  the  actual,  measured  distance  of 
a  sensor  beam  to  its  expected  distance  computed  by  ray  tracing,  under  the  assumption  that  the  robot’s  pose  is  s.  A  comparison 
of  actual  data  and  our  (learned)  mixture  model  shows  good  correspondence.  Diagram  (b)  shows  a  specific  laser  range  scan  o,  for 
which  diagram  (c)  plots  the  density  p(o|s,  m)  for  different  locations  in  the  map. 


Figure  3  illustrates  global  mobile  robot  localization  based  on  sonar  measurements  in  an  office  environ¬ 
ment,  using  our  robot  Rhino.  The  robot’s  path  is  outlined  in  the  first  diagram,  along  with  four  reference 
locations.  Shown  there  is  also  the  initial  belief,  which  is  uniform,  since  the  robot  does  not  know  where  it  ' 
is.  The  posterior  belief  after  moving  from  the  first  to  the  second  reference  location  is  shown  in  Figure  3b. 

At  this  point,  most  of  the  probability  mass  is  located  in  the  corridor,  but  the  robot  still  does  not  know  where 
it  is.  This  diagram  illustrates  nicely  one  of  the  features  of  the  probabilistic  approach,  namely  its  ability  to 
pursue  multiple  hypotheses,  weighted  by  sensor  evidence.  After  moving  to  the  third  reference  position,  the 
belief  is  centered  around  two  discrete  locations,  as  shown  in  Figure  3c.  Finally,  after  moving  into  one  of  the 
rooms,  the  symmetry  is  broken  and  the  robot  is  highly  certain  as  to  where  it  is  (Figure  3d). 

Of  fundamental  importance  for  the  design  of  probabilistic  algorithms  is  the  choice  of  the  representation. 

One  of  the  most  powerful  approximations  is  known  as  importance  sampling  [74],  and  the  resulting  algorithm 
is  known  under  names  like  particle  filters  [22, 23,  61 , 68],  condensation  algorithm  [42, 43]  and  Monte  Carlo 
localization  [19,  29];  here  we  refer  to  it  as  Monte  Carlo  localization  (MCL).  The  basic  idea  of  MCL  is  to 
approximate  b(s)  with  a  weighted  set  of  samples  (particles),  so  that  the  discrete  distribution  defined  by 
the  samples  approximates  the  desired  one.  The  weighting  factors  are  called  importance  factors  [74].  The 
initial  belief  is  represented  by  a  uniform  sample  of  size  rn,  that  is,  a  set  of  m  samples  drawn  uniformly 
from  the  space  of  all  poses,  annotated  by  the  constant  importance  factor  m“^.  MCL  implements  the  update 
equation  (4)  by  constructing  a  new  sample  set  from  the  current  one  in  response  to  an  action  item  and 
an  observation 

1.  Draw  a  random  sample  from  the  current  belief6/_i(.S/_i),  with  a  likelihood  given  by  the  importance 
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Figure  3:  Example  of  grid-based  Markov  localization  in  a  symmetric  environment,  (a)  robot  path,  highlighting  four  robot  poses, 
(b)  to  (d):  belief  at  2nd,  3rd,  and  4th  pose,  respectively. 


factors  of  the  belief  (  sf-i ). 

2.  For  this  St-i,  guess  a  successor  pose  Sf,  according  to  the  distribution  p(sf|ay_i ,  m). 

3.  Assign  a  preliminary  importance  factor  p(of  |.s<,  m)  to  this  sample  and  add  it  to  the  new  sample  set  repre¬ 
senting  bf{s-t). 

4.  Repeat  Step  1  through  Zm  times.  Finally,  normalize  the  importance  factors  in  the  new  sample  set  hf{st) 
so  that  they  add  to  1 . 

Figure  4  shows  MCL  in  action.  Shown  in  the  first  diagram  is  a  belief  distribution  (sample  set)  at  the 
beginning  of  the  experiment  when  the  robot  does  not  (yet)  know  its  position.  Each  dot  is  a  three-dimensional 
sample  of  the  robot’s  a  -y-location  along  with  its  heading  direction  0.  The  second  diagram  shows  the  belief 
after  a  short  motion  segment,  incorporating  several  sonar  readings.  At  this  point,  most  samples  concentrate 
on  two  locations;  however,  the  symmetry  of  the  corridor  makes  it  impossible  to  disambiguate  them.  Finally,  ' 
the  third  diagram  in  Figure  4  shows  the  belief  after  the  robot  moves  into  one  of  the  rooms,  enabling  it  to 
disambiguate  its  location  with  high  confidence. 

The  MCL  algorithm  is  in  fact  quite  efficient;  slight  modifications  of  the  basic  algorithms  [56,  92]  re¬ 
quire  as  few  as  100  samples  for  reliable  localization,  consuming  only  a  small  fraction  of  time  available 
on  low-end  PC.  It  can  also  be  implemented  as  any -time  algorithm  [17,  95],  meaning  that  it  can  adapt  to 
the  available  computational  resources  by  dynamically  adjusting  the  number  of  samples  m.  With  slight 
modifications — such  as  sampling  from  the  observation  [92] — ^MCL  has  been  shown  to  recover  gracefully 
from  global  localization  failures,  such  as  manifested  in  the  kidnapped  robot  problem  [27],  where  a  well- 
localized  robot  is  teleported  to  some  random  location  without  being  told.  For  these  reasons,  probabilistic 
algorithms  like  MCL  are  currently  the  best  known  methods  for  such  hard  localization  problems. 

Another  feature  of  MCL  is  that  its  models — in  particular  p(s|a,  s,  m),p{o\s,  m)  and  the  map — can  be 
extremely  crude  and  simplistic,  since  probabilistic  models  carry  their  own  notion  of  uncertainty.  This  makes 
probabilistic  algorithms  relatively  easy  to  code.  In  comparison,  traditional  robotics  algorithms  that  rely  on 
deterministic  models  make  much  stronger  demands  on  the  accuracy  of  the  underlying  models. 
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Figure  4:  Global  localization  of  a  mobile  robot  using  MCL. 


3  Mapping 


A  second  area  of  robotics,  where  probabilistic  algorithms  have  proven  remarkably  successful,  is  mapping. 
Mapping  is  the  problem  of  generating  maps  from  sensor  measurements.  This  estimation  problem  is  much 
higher-dimensional  than  the  robot  localization  problem— in  fact,  in  its  pure  form  one  could  argue  the  prob¬ 
lem  possesses  infinitely  many  dimensions.  What  makes  this  problem  particularly  difficult  is  its  chicken- 
and-egg  nature,  which  arises  from  the  fact  that  position  errors  accrued  during  mapping  are  difficult  to  com¬ 
pensate  [71].  Put  differently,  localization  with  a  map  is  relatively  easy,  as  is  mapping  with  known  locations. 
In  combination,  however,  this  problem  is  hard. 

This  section  will  review  three  major  paradigms  in  mobile  robot  mapping,  all  of  which  are  probabilistic 
and  follow  from  the  same  mathematical  framework.  Let  us  begin  by  the  most  obvious  idea,  which  is  the 
idea  of  using  the  same  approach  for  mapping  as  for  localization.  If  we  augment  the  state  s  that  is  being 
estimated  via  Equation  (4)  by  the  map  nit — ^the  subscript  t  indicates  that  we  allow  the  map  to  change  over 
time — ^Equation  (4)  becomes 


bt{si,mt)  =  Pf  p{ot\st,mt)  J  ^ p(.s/,  s,_i,  m/_i) 


6t_i(s#_i,??7f_i)  dst-i  dnit-i.  (5) 


If  the  map  is  assumed  to  be  static — which  is  commonly  assumed  throughout  the  literature — ^the  maps  at 
times  t  and  /  —  1  will  be  equivalent.  This  implies  that  mt\at-i,  Sf-i,  mt-\)  is  zero  if  nit  7^  and 
.S/_1,  ?77(._i)  if  777*  =  nit-1.  The  integration  over  maps  in  (5)  is  therefore  eliminated,  yielding 

hi{st,m)  =  p{ot\st,ni)  I  p{st\ai-\,st-i,ni)ht-i{st-i,m)  dst^i.  (6) 


The  major  problem  with  (6)  is  complexity.  The  belief  tii)  is  a  density  function  in  a  (A'  +3) -dimensional 
space,  where  N  is  the  number  of  free  parameters  that  constitute  a  map  (e.g.,  a  constant  times  the  number 
of  landmarks),  and  the  additional  3  parameters  specify  the  robot’s  pose.  N  can  be  very  large  (e.g.,  1,000), 
which  makes  the  posterior  estimation  problem  hard.  To  make  matters  worse,  the  belief  m)  cannot 
easily  be  factorized,  since  the  uncertainty  of  map  items  and  robot  poses  are  often  highly  correlated  [83], 

The  most  successful  attempt  to  implement  (6)  employs  Kalman  filters  [13, 14, 57, 58],  which  goes  back 
to  a  semina]  paper  by  Smith,  Self,  and  Cheeseman  [83].  Recall  that  Kalman  filters  represent  beliefs  by 
Gaussians;  thus,  they  require  0{N‘^)  parameters  to  represent  the  posterior  over  an  A^ -dimensional  space. 
Calculating  (6)  involves  matrix  inversion,  which  can  be  done  in  0(A^^)  time  [63].  This  critically  limits 
the  number  of  features  that  can  be  mapped  (see  [59]  for  a  recent  attempt  to  escape  this  limitation  using 
hierarchies  of  maps).  In  practice,  this  approach  has  been  applied  to  mapping  several  hundreds  of  free 
parameters  [57]. 

The  basic  Kalman  filtering  approach  to  mapping  is  also  limited  in  a  second,  more  important  way.  In 
particular,  it  requires  that  features  in  the  environment  can  be  uniquely  identified — which  is  a  consequence 
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of  the  Gaussian  noise  assumption.  For  example,  it  does  not  suffice  to  know  that  the  robot  faces  a  door; 
instead,  it  must  know  which  door  it  faces,  to  establish  correspondence  to  previous  sightings  of  the  same 
door.  This  limitation  is  of  great  practical  importance.  It  is  common  practice  to  extract  a  small  number 
of  identifiable  features  from  the  sensor  data,  at  the  risk  of  discarding  all  other  information.  Some  recent 
approaches  overcome  this  assumption  by  “guessing”  the  correspondence  between  measurements  at  different 
points  in  time,  but  they  tend  to  be  brittle  if  those  guesses  are  wrong  [36,  62].  However,  if  the  assumptions 
are  met,  Kalman  filters  generate  optimal  estimates,  and  in  particular  they  outperform  any  non-probabilistic 
approach. 

An  alternative  approach,  proposed  in  [91],  seeks  to  estimate  the  mode  of  the  posterior,  argmax,,,  b(m), 
instead  of  the  full  posterior  b{m).  This  might  appear  quite  modest  a  goal  compared  to  the  full  posterior 
estimation.  However,  if  the  correspondence  is  unknown  (and  noise  is  non-Gaussian),  this  in  itself  is  a 
challenging  problem.  To  see,  we  note  that  the  posterior  over  maps  can  be  obtained  in  closed  form: 


bi(m) 


p(m\do...t)  =  Jbi{si,m)dsf 

i  1 

r=0  T=l 


ar-i,-9r-um)  dsi  d92  ...  dSi, 


(7) 


where  the  initial  pose  is — somewhat  arbitrarily — set  to  =  (0,0,0).  This  expression  is  obtained  from  (6) 
by  integrating  over  S/,  followed  by  recursive  substitution  of  the  belief  from  time  /  —  1  to  time  0,  and  resorting 
of  the  resulting  terms  and  integrals.  For  convenience,  we  will  assume  a  uniform  prior  p[m.),  transforming 
the  problem  into  a  maximum  likelihood  estimation  problem.  Notice  that  Equation  (7)  integrates  over  all 
possible  paths,  a  rather  complex  integration.  Unfortunately,  we  know  of  no  way  to  calculate  argmax, „  bf  [m  ) 
analytically  for  data  sets  of  reasonable  size. 

To  find  a  solution,  we  notice  that  the  robot’s  path  can  be  considered  “missing  variables”  in  the  optimiza¬ 
tion  problem;  knowing  them  indeed  greatly  simplifies  the  problem.  The  statistical  literature  has  developed  a 
range  of  algorithms  for  such  problems,  one  of  which  is  the  EM  algorithm  [20,  64].  This  algorithm  computes 
a  sequence  of  maps,  denoted  . . . ,  which  successively  increasing  likelihood.  The  superscript 

is  not  to  be  confused  with  the  time  index  t  or  the  index  of  a  particle  /;  all  it  refers  to  is  the  iteration  of  the 
optimization  algorithm. 

EM  calculates  a  new  map  by  iterating  two  steps,  an  expectation  step,  or  E-step,  and  a  maximization  step, 
or  M-step: 


•  In  the  E-step,  EM  calculates  an  expectation  of  a  joint  log-likelihood  function  of  the  data  and  the  poses, 
conditioned  on  the  ^"-th  map  (and  conditioned  on  the  data): 


Q[m\m^^^]  =  [log ;;(.so.  •  | 


(8) 


This  might  appear  a  bit  cryptic,  but  the  key  thing  here  is  that  computing  Q  involves  calculating  the 
posterior  distribution  over  poses  sq.  ....  Sf  conditioned  on  the  k-th  model  777  W.  This  is  good  news,  as 
we  have  already  seen  how  to  estimate  the  posterior  over  poses  given  a  map.  Technically,  calculating  (8) 
involves  two  localization  runs  through  the  data,  a  forwards  run  and  a  backwards  run,  since  all  data  has 
to  be  taken  into  account  when  computing  the  posterior  p(s>|f/o.../)  (the  algorithm  above  only  considers 
data  up  to  time  r).  We  also  note  that  in  the  very  first  iteration,  we  do  not  have  a  map.  Thus,  Q[m]m\^^] 
calculates  the  posterior  for  a  “blind”  robot,  i.e.,  a  robot  that  ignores  its  measurements  oi , . . . ,  O/. 

•  In  the  M-step,  the  most  likely  map  is  computed  given  the  pose  estimates  obtained  in  the  E-step.  This  is 
formally  written  as 

_  argmax  (5  [777 1 777  (9) 
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Figure  5:  (a)  Raw  data  of  a  large  open  hall  (the  Dinosaur  Hall  in  the  Carnegie  Museum  of  Natural  History,  Pittsburgh,  PA)  and  (b) 
map  constructed  using  EM  and  occupancy  grid  mapping,  (c)  Occupancy  grid  map  of  another  museum  (The  Tech  Museum  in  San 
Jose,  CA),  and  (d)  architectural  blueprint  for  comparison. 


Technically,  this  is  still  a  very  difficult  problem,  since  it  involves  finding  the  optimum  in  a  high-dimensional 
space.  However,  it  is  common  practice  to  decompose  the  problem  into  a  collection  of  one-dimensional 
maximization  problems,  by  stipulating  a  grid  over  the  map  and  solving  (9)  independently  for  each  grid 
cell.  The  maximum  likelihood  estimation  for  the  resulting  single-cell  random  variables  is  mathematically 
straightforward. 

Iterations  of  both  steps  tends  to  increase  the  likelihood  (currently  we  lack  a  proof  of  convergence  due  to 
the  decomposition  in  the  M-step).  However,  we  found  that  this  approach  works  very  well  in  practice  [91], 
solving  hard  mapping  problems  that  were  previously  unsolved  (see  also  [77,  78]). 

The  decomposition  in  the  M-step  is  quite  common  for  mapping  algorithms  that  assume  knowledge  of  the 
robot’s  pose.  It  goes  back  to  the  seminal  work  by  Elfes  and  Moravec  on  occupancy  grid  mapping  [26,  66], 
a  probabilistic  algorithm  that  is  similar,  though  not  identical,  to  the  M-step  above.  This  brings  us  to  the 
third  mapping  algorithm.  Occupancy  grid  mapping  is  currently  the  most  widely  used  mapping  algorithm  for 
mobile  robots  [3, 25,  37,  88, 93],  often  augmented  by  ad-hoc  methods  for  localization  during  mapping.  It  is 
another  prime  example  for  the  success  of  probabilistic  algorithms  in  robotics. 

Occupancy  grid  mapping  addresses  a  much  simpler  problem  that  the  one  above,  namely  the  problem  of 
estimating  a  map  from  a  set  of  sensor  measurements  given  that  one  already  knows  the  corresponding  poses. 
Let  {x,  y)  denote  a  specific  grid  cell,  and  be  the  random  variable  the  models  its  occupancy  at  time  t. 

Occupancy  is  a  binary  concept;  thus,  we  will  write  if  a  cell  is  occupied,  and  if  it  is  not. 
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Substituting  m 


i^'y) 

t 


into  Equation  (4)  under  the  consideration  that  this  is  a  discrete  random  variable  yields 


bfim 


iry)^ 


Vt 


(10) 


which  in  static  worlds  can  be  simplified  to 


(11) 


The  second  transformation  pays  tribute  to  the  fact  that  in  occupancy  grid  mapping,  one  often  is  given 
p(^mi'^-y)\o^)  instead  of  One  could  certainly  leave  it  at  this  and  calculate  the  normalization 

factor  i]f  at  run-time.  However,  for  binary  random  variable  the  noimalizer  can  be  eliminated  by  noticing  that 
the  so-called  odds,  which  is  the  following  quotient: 


6/(777.^^’^)— 0)  77(  777^^’^^^)  =  0|O/)  /)( ??7X‘'^’i/)  =  1 )  (777^^’^)=0) 

As  is  easily  shown  [88],  this  expression  has  the  closed -form  solution 

p(7^7{-i'iy)_i) 


(12) 


6^(777<^’'^)  =  1)  =  1~<1  + 


1  —  77(777<'^^^^)=:1) 


^  ^^(y77{a\v>_i|^^)  2  —  77(777^-^^^^^  =  1) 

1  —  77  (  777  =  1 1  Ot-  )  77 (  777  —  1 ) 


-=0 


All  three  of  these  algorithms  have  shown  to  be  highly  robust  and  accurate  in  practice,  and  there  are  among 
the  best  algorithms  in  existence.  For  example.  Figure  5a  shows  a  raw  data  set  of  a  large  hall  (approximately 
50  meters  wide),  along  with  the  result  of  first  applying  EM,  and  then  occupancy  grid  mapping  using  the 
poses  estimated  with  EM  (Figure  5b).  The  map  in  Figure  5c  has  been  generated  using  a  similar  probabilistic 
algorithm  that  runs  on-line  (unlike  EM)  [90]  (see  also  [35]);  Figure  5d  shows  an  architectural  blueprint  for 
comparison.  Cyclic  environments  are  among  the  most  difficult  ones  to  map,  since  the  odometry  error  can 
be  very  large  when  closing  the  cycle.  These  results  illustrate  that  EM  and  occupancy  grid  mapping  yield 
excellent  results  in  practice.  While  the  maps  shown  here  are  two-dimensional,  probabilistic  algorithms  have 
also  successfully  been  applied  to  build  three-dimensional  maps  [90], 

These  results  illustrate  that  probabilistic  algorithms  are  well  suited  for  high-dimensional  estimation  and 
learning  problems;  in  fact,  we  know  of  no  comparable  algorithm  that  can  solve  problems  of  equal  hardness 
but  does  not  explicitly  address  the  inherent  uncertainty  in  perception.  To  date,  the  best  mapping  algorithms 
are  probabilistic,  and  most  of  them  are  versions  of  the  three  algorithms  described  above.  Our  analysis 
also  suggests  that  probabilistic  algorithms  are  somewhat  of  a  natural  fit  for  problems  like  the  ones  studied 
here.  Past  research  has  shown  that  many  estimation  and  learning  problems  in  robotics  have  straightforward 
solutions  when  expressed  using  the  language  of  probability  theory,  with  mapping  just  being  one  example. 


4  Robot  Control 

Finally,  let  us  turn  our  attention  to  the  issue  of  robot  control.  The  ultimate  goal  of  robotics  is  to  build  robots 
that  do  the  right  thing.  As  stated  in  the  introduction,  we  conjecture  that  a  robot  that  takes  its  own  uncertainty 
into  account  when  selecting  actions  will  be  superior  to  one  that  does  not. 

Unfortunately,  the  field  of  probabilistic  robot  control  is  much  poorer  developed  than  that  of  probabilistic 
perception.  This  is  because  of  the  enormous  computational  complexity  of  decision  making.  However,  within 
AI  this  issue  has  recently  received  considerable  attention.  Even  in  robotics,  some  noticeable  successes  have 
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been  achieved,  where  probabilistic  algorithms  outperformed  conventional,  non-probabilistic  algorithms  [45, 
81]. 

One  such  algorithm  is  the  coastal  navigation  algorithm  [72,  73]  (see  also  [45]),  a  motion  planning 
algorithm  for  mobile  robots  that  takes  uncertainty  into  account.  The  algorithm  was  originally  motivated  by 
the  observation  that  ships  that  navigate  through  open  water  without  GPS  often  stay  in  close  proximity  to  the 
coast,  to  reduce  the  danger  of  getting  lost.  The  same  applies  to  mobile  robots:  The  choice  of  control  can 
have  a  profound  impact  on  the  likelihood  of  localization  errors.  The  coastal  navigation  algorithm  selects 
paths  accordingly,  explicitly  considering  uncertainty. 

To  study  this  algorithm,  let  us  step  back  a  little  and  consider  the  mathematical  framework  that  underlies 
this  and  many  other  probabilistic  control  algorithms:  POMDPs.  POMDP  (short  for  partially  ohseiyahle 
Markov  decision  processes)  is  a  framework  for  acting  optimally  under  uncertainty  in  sequential  decision 
tasks.  While  POMDPs  can  be  traced  back  to  the  Seventies  [65,  82,  84],  the  AI  community  has  only  recently 
begun  to  pay  attention  to  this  framework,  motived  by  the  work  by  Littman,  Cassandra,  and  Kaelbling  [46, 
60].  POMDPs  address  the  problem  of  choosing  actions  so  as  to  minimize  a  scalar  (immediate)  cost  function, 
denoted  C{s).  For  example,  in  robot  motion  planning,  one  might  set  C(.s)  =  0  for  goal  locations  s,  and 
-1  elsewhere.  Since  reaching  a  goal  location  typically  requires  a  whole  sequence  of  actions,  the  control 
objective  is  to  minimize  the  expected  cumulative  cost: 

t+T 

J  =  E[C{Sr)].  (14) 

r=f+l 

Here  the  expectation  is  taken  over  all  future  states.  T  may  be  oo — in  which  case  cost  is  often  discounted 
over  time  by  an  exponential  factor. 

Many  important  POMDP  algorithms  [46,  60]  are  off-line  algorithms,  in  the  sense  that  they  calculate  a 
policy  for  action  selection  for  arbitrary  situations  (i.e.,  belief  states)  in  an  explicit,  off-line  phase.  The  policy 
is  denoted  tt  and  maps  belief  states  into  actions. 

The  most  prominent  approach  to  calculating  tt  is  value  iteration  [1, 41],  a  version  of  dynamic  program¬ 
ming  for  computing  the  expected  cumulative  cost  of  belief  states  that  has  become  highly  popular  in  the  field 
of  reinforcement  learning  [47,  85].  Value  iteration  in  belief  space  computes  a  value  function,  denoted  by 
V,  which  in  the  ideal  case  measures  the  expected  cumulative  cost  if  one  starts  in  a  state  s  drawn  according 
to  the  belief  distribution  b,  and  acts  optimally  thereafter.  Thus,  the  value  V{b)  of  the  belief  state  is  the  best 
possible  cumulative  costs  one  can  expect  for  being  in  b.  This  is  expressed  as 

tff-T 

=  /  II  E[C{sr)\st  =  s]b{s)  ds.  (15) 

T=t+1 

Following  [  1 , 85],  the  value  function  can  be  computed  by  recursively  adjusting  the  value  of  individual  belief 
states  b  according  to 

V(b)  i —  min  J [V{b')  C{b')]  p{b'\a,  b,  m)  db\  (16) 

which  assigns  to  V{b)  the  expected  value  at  the  next  belief,  b'.  Here  the  immediate  cost  of  a  belief  state  V  is 
obtained  by  integrating  over  all  states  C{b')  =  /  C{s')  b'{s')  ds'.  The  conditional  distribution p(6'|a,  b,  m) 
is  the  belief  space  counterpart  to  the  next  state  distribution,  which  is  obtained  as  follows: 

p{b'\a,b,m)  =  f  p{b'\o',a,b,m)  p{o'\a,b,m)  do',  (17) 
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(a) 


(b) 


Figure  6:  Coastal  plans:  the  robot  actively  seeks  the  proximity  of  obstacles  to  improve  its  localization.  The  large  open  area  in  the 
center  of  this  Smithsonian  museum  is  approximately  20  meters  wide  and  usually  crowded  with  people. 


where  p{b^\o\  a,  b,  m)  is  a  Dirac  distribution  defined  through  Equation  (4),  and 

p{o'\a,  6,  m)  —  J  J  p{o^\s\  111)  p{s'\a^  s,  m)  b{s)  ds'  ds.  (18) 

Once  V  has  been  computed,  the  optimal  policy  is  obtained  by  selecting  actions  that  minimize  the  expected 
V’“value  over  all  available  actions: 

7r(6)  —  argmin  f  V {b')  p{b'\a,b^  m)  db\  (19) 

a  J 

While  this  approach  defines  a  mathematically  elegant  and  consistent  way  to  compute  the  optimal  policy 
from  the  known  densities  p{s^\o^  s,  m)  and  p[o^\s\  m) — which  are  in  fact  the  exact  same  densities  used  in 
MCL — there  are  two  fundamental  problems.  First,  in  continuous  domains  the  belief  space  is  the  space  of  all 
distributions  over  the  continuum,  which  is  an  infinitely-dimensional  space.  Consequently,  no  exact  method 
exists  for  calculating  V  in  the  general  case.  Second,  even  if  the  state  space  is  discrete — which  is  commonly 
assumed  in  the  POMDP  framework — the  computational  burden  can  be  enormous.  This  is  because  for  state 
spaces  of  size  ??.,  the  corresponding  belief  space  is  a  (??  -  1) -dimensional  continuous  space.  The  best  known 
solutions,  such  as  the  Mitness  algorithm  [46],  can  only  handle  problems  of  the  approximate  size  of  100  states, 
and  a  planning  horizon  of  no  more  than  T  =  5  steps.  In  contrast,  state  spaces  in  robotics  routinely  possess 
orders  of  magnitude  more  states,  even  under  crude  discretizations.  This  makes  approximating  imperative. 

Coastal  navigation  is  an  extension  of  POMDPs  that  relies  on  an  approximate  representation  for  belief 
states  b.  The  underlying  assumption  is  that  the  exact  nature  of  the  uncertainty  is  irrelevant  for  action  selec¬ 
tion;  instead,  it  suffices  to  know  the  degree  of  uncertainty  as  expressed  by  the  entropy  of  a  belief  state  H[b], 
Thus,  coastal  navigation  represents  belief  states  by  the  following  tuple: 


b  =  ( argmax6(.S');  H[b]),  (20) 

s 

While  this  approximation  is  coarse,  it  causes  value  iteration  to  scale  exponentially  better  to  large  state  spaces 
than  the  full  POMDP  solution,  while  still  exhibiting  good  performance  in  practice. 

Figure  6  shows  an  example  trajectory  calculated  by  the  coastal  navigation  algorithm  for  a  large,  fea¬ 
tureless  environment:  a  Smithsonian  museum  in  Washington,  DC.  The  goal  of  motion  is  to  reach  a  target 
location  with  high  probability.  By  considering  uncertainty,  the  coastal  planner  generates  paths  that  actively 
seeks  the  proximity  of  known  obstacles  so  as  to  minimize  the  localization  error — at  the  expense  of  an  in¬ 
creased  path  length  when  compared  to  the  shortest  path.  Experimental  results  [73]  have  shown  that  the 
success  rate  of  the  coastal  planner  is  superior  to  conventional  shortest  path  planners  that  ignore  the  inherent 
uncertainty  in  robot  motion. 


11 


Figure  7;  Probabilistic  algorithms  were  used  pervasively  for  the  museum  tour-guide  robots  Rhino  (left  image)  and  Minerva  (right 
two  images),  which  guided  thousand  of  people  through  crowded  museums  fully  autonomously. 

Coastal  navigation  is  only  one  out  of  many  examples.  It  highlights  an  important  principle,  namely 
that  crude  approximations  can  go  a  long  way  when  implementing  probabilistic  control  algorithms.  Recent 
research  had  led  to  a  range  of  other  control  algorithms  that  rely  on  approximate  probabilistic  representa¬ 
tions.  Of  particular  importance  are  algorithms  for  maximizing  knowledge  gain,  which  typically  rely  on  a 
single-step  search  horizon  to  generate  robot  control.  Examples  include  the  rich  work  on  robot  exploration, 
in  which  robots  (or  teams  thereof)  select  actions  so  as  to  maximally  reduce  their  uncertainty  about  their 
environments  [24,  50,  54,  79,  88,  94].  They  also  include  work  on  active  localization  [11,  30],  where  a  robot 
moves  to  places  that  maximally  disambiguate  its  pose.  Another  class  of  approaches  rely  on  tree  search  for 
policy  determination,  such  as  the  work  on  active  perception  and  sensor  planning  by  Kristensen  [52,  53]. 
His  approach  uses  models  of  uncertainty  to  select  the  appropriate  sensors  in  an  indoor  navigation  task.  All 
these  approaches  have  demonstrated  that  probabilistic  algorithms  lead  to  more  robust  solutions  to  important 
control  problems  in  robotics. 

5  A  Case  Study:  Museum  Tour-Guide  Robots 

Probabilistic  algorithms  have  been  at  the  core  of  a  number  of  state-of-the-art  robot  systems  (see  e.g.,  [2, 21]), 
such  as  the  Xavier  robot  mentioned  above  [80].  In  fact,  recently  the  number  of  publications  on  statistically 
sound  algorithms  for  perception  and  control  has  increased  dramatically  at  leading  robotics  conferences. 

In  our  own  work,  we  recently  developed  two  autonomous  museum  tour-guide  robots  [8,  89]  (see 
also  [40]),  which  pervasively  employed  probabilistic  algorithms  for  navigation  and  people  interaction.  Pic¬ 
tures  of  both  robots  are  shown  in  Figure  7.  The  robot  shown  on  the  left.  Rhino,  was  the  world’s  first  museum 
tour-guide  robot,  which  was  deployed  at  the  Deutsches  Museum  in  Bonn  in  1997.  The  other  robot,  Minerva, 
led  thousands  of  people  through  a  crowded  Smithsonian  museum  in  Washington,  DC.  Both  robots  were 
developed  to  showcase  probabilistic  algorithms  in  complex  and  highly  dynamic  environments.  They  were 
unique  in  their  ability  to  navigate  safely  and  reliably  in  the  proximity  of  people. 

The  task  of  these  robots  was  simple:  to  attract  people,  interact  with  them,  and  guide  them  from  exhibit 
to  exhibit.  Several  factors  made  this  problem  challenging.  To  find  their  way  around,  the  robots  had  to  know 
where  they  were.  However,  large  crowds  of  people  almost  permanently  blocked  the  robots’  sensors,  making 
localization  a  difficult  problem.  In  fact,  people  frequently  sought  to  confuse  the  robot,  or  force  it  close  to 
hazards  such  as  downward  staircases.  To  make  matters  worse,  the  robots’  ability  to  sense  such  hazards  was 
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extremely  limited.  For  example,  the  sensors  consistently  failed  to  sense  glass  cases  put  up  for  the  protection 
of  certain  exhibits,  and  neither  robot  possessed  a  sensor  to  detect  staircases.  Thus,  accurate  localization 
played  a  prime  role  in  avoiding  collisions  with  such  “invisible”  obstacles  and  hazards  such  as  staircases, 
whose  location  was  modeled  in  the  map. 

To  challenge  the  autonomy  of  our  robots,  we  did  not  modify  the  environment  in  any  way.  And  even 
though  the  museums  were  crowded,  the  robots  had  to  navigate  at  approximate  walking  speeds  to  sustain 
people’s  interest,  while  avoiding  collisions  with  people  at  all  costs. 

Detailed  descriptions  of  the  robots’  software  architecture  and  experimental  findings  are  beyond  the  scope 
of  this  paper  (see  [8,  89]);  here  we  remark  that  probabilistic  algorithms  were  employed  at  virtually  all  levels 
of  the  software  architecture.  In  total,  both  robots  traversed  a  distance  of  more  than  60km,  with  average 
speeds  between  30cm/sec  and  80cm/sec,  and  top  speeds  well  above  160cm/sec.  In  Rhino’s  case,  we  carefully 
evaluated  every  failure,  and  concluded  that  only  one  major  localization  failure  was  observed  over  a  period 
of  six  days  [8];  however,  this  localization  failure  coincided  with  a  malfunctioning  of  the  sonar  sensors. 
Rhino  employed  a  probabilistic  collision  avoidance  routine  that  guaranteed,  with  high  likelihood,  that  the 
robot  does  not  collide  with  “invisible”  obstacles  even  when  the  robot  is  highly  uncertain  where  it  is  [31]. 
In  addition,  Minerva  employed  probabilistic  algorithms  to  learn  occupancy  grid  maps  of  the  museums.  In 
other  experiments,  we  have  devised  practical  probabilistic  algorithm  for  active  exploration,  both  in  pursuit 
of  finding  out  where  a  robot  is  [1 1]  and  learning  maps  of  unknown  terrain  [88]  with  teams  of  collaborating 
robots  [10], 

In  all  these  cases,  the  probabilistic  nature  of  the  algorithms  has  been  essential  for  achieving  robustness 
in  the  face  of  uncertain  and  dynamic  environments.  And  all  these  algorithms  rely  on  sometimes  remarkably 
simple  approximations  and  shortcuts  that  make  hard  problems  computationally  tractable. 


6  Discussion 

The  last  few  decades  have  led  to  a  flurry  of  different  software  design  paradigms  for  autonomous  robots. 
Early  work  on  model-based  robotics  often  assumed  the  availability  of  a  complete  and  accurate  model  of  the 
robot  and  its  environment,  relying  on  planners  (or  theorem  provers)  to  generate  actions  [12,  55,  76].  Such 
approaches  are  often  inapplicable  to  robotics  due  to  the  difficulty  of  generating  models  that  are  sufficiently 
accurate  and  complete.  Recognizing  this  limitation,  some  researchers  have  advocated  model-free  reactive 
approaches.  Instead  of  relying  on  planning,  these  approaches  require  programmers  to  program  controllers 
directly.  A  popular  example  of  this  approach  is  the  “subsumption  architecture”  [7],  where  controllers  are 
composed  of  small  finite  state  automata  that  map  sensor  readings  into  control  while  retaining  a  minimum 
of  internal  state.  Some  advocates  of  this  approach  went  as  far  as  refusing  the  need  for  internal  models 
and  internal  state  altogether  [7,  15].  Observing  that  ''the  world  is  its  own  best  moder  [6],  behavior-based 
approaches  usually  rely  on  immediate  sensor  feedback  for  determining  a  robot’s  action.  Obvious  limits  in 
perception  (e.g.,  robots  can’t  see  through  walls)  pose  clear  boundaries  on  the  type  tasks  that  can  be  tackled 
with  this  approach.  This  leads  us  to  conclude  that  while  the  world  might  well  be  its  most  accurate  model,  it 
is  not  necessarily  its  most  accessible  one  [86].  And  accessibility  matters! 

The  probabilistic  approach  is  somewhere  between  these  two  extremes.  Probabilistic  algorithms  rely  on 
models,  just  like  the  classical  plan-based  approach.  For  example,  Markov  localization  requires  a  perception 
model  p(o|.s*,  777),  a  motion  model  77(cs'|a,  6*),  and  a  map  of  the  environment.  However,  since  these  models 
are  probabilistic,  they  only  need  to  be  approximate.  This  makes  them  much  easier  to  implement  (and 
to  learn!)  than  if  we  had  to  meet  the  accuracy  requirements  of  traditional  approaches.  Additionally,  the 
ability  to  acknowledge  existing  uncertainty  and  even  anticipate  upcoming  uncertainty  in  planning  leads  to 
qualitatively  new  solutions  in  a  range  of  robotics  problems,  as  demonstrated  in  this  article. 
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Probabilistic  algorithms  are  similar  to  behavior-based  approaches  in  that  they  place  a  strong  emphasis 
on  sensor  feedback.  Because  probabilistic  models  are  insufficient  to  predict  the  actual  state,  sensor  measure¬ 
ments  play  a  vital  role  in  state  estimation  and,  thus,  in  determining  a  robot’s  actual  behavior.  However,  they 
differ  from  behavior-based  approaches  in  that  they  rely  on  planning,  and  in  that  a  robot’s  behavior  is  not 
just  a  function  of  a  small  number  of  recent  sensor  readings.  As  an  example  that  illustrates  the  importance 
of  the  latter,  imagine  placing  a  mobile  robot  in  an  a  crowded  place  full  of  invisible  hazards!  Surely,  the 
problem  can  be  solved  by  adding  more  sensors;  however,  such  an  approach  is  expensive  at  best,  but  more 
often  it  will  be  plainly  infeasible  due  to  lack  of  appropriate  sensors.  Our  robot  Rhino,  for  example,  was 
equipped  with  five  different  sensor  systems — vision,  laser,  sonar,  infrared  and  tactile — ^yet  it  still  could  not 
perceive  all  the  hazards  and  obstacles  in  this  fragile  environment  with  the  necessary  reliability  (see  [8]  for 
a  discussion).  Thus,  it  seems  unlikely  that  a  reactive  approach  could  have  performed  anywhere  nearly  as 
reliably  and  robustly  in  this  task. 

Probabilistic  robotics  is  closely  related  to  a  rich  body  of  literature  on  uncertainty  in  AI  ([39,  67]  are 
good  starting  points,  as  are  any  recent  proceedings  of  UAI).  In  fact,  many  of  the  basic  algorithms  in  robotics 
have  counterparts  in  the  UAI  community,  the  major  difference  being  that  their  focus  tends  be  on  discrete 
spaces,  whereas  robots  typically  live  in  continuous  spaces.  Also,  building  real  robotics  systems  constrains 
the  assumptions  under  which  one  can  operate.  Consequently,  approximations  and  real-time  algorithms  play 
a  greater  role  in  robotics  than  they  play  currently  in  mainstream  AI. 

One  of  the  most  exciting  aspect  of  the  probabilistic  paradigm  is  that  it  opens  great  new  opportunities  for 
basic  research  in  robotics  and  AI,  with  great  potential  for  high  impact  in  robotics  and  beyond.  Probabilistic 
algorithms  are  still  far  from  mainstream  in  robotics,  and  there  is  a  range  of  problems  that  appear  to  be 
highly  amenable  to  probabilistic  solutions.  We  therefore  conclude  this  article  by  laying  out  five  broad  areas 
of  research  which  we  deem  to  be  highly  important. 

•  Representations.  The  choice  of  representation  is  crucial  in  the  design  of  any  probabilistic  algorithm, 
as  it  determines  its  robustness,  efficiency,  and  accuracy.  Recent  research  has  already  led  to  a  range  of 
representations  for  probabilistic  information  in  continuous  spaces,  such  as  the  particle  representation  in 
the  example  described  above.  However,  the  development  of  new  representations  is  absolutely  essential 
for  scaling  up  to  more  complex  problems,  such  as  the  control  of  highly  articulated  robots  or  multi-robot 
coordination. 

•  Learning.  The  probabilistic  paradigm  lends  itself  naturally  to  learning,  yet  very  little  work  has  been 
carried  out  on  automatically  learning  models  (or  behaviors)  in  real-world  robotic  applications  using  prob¬ 
abilistic  representations.  Many  of  today’s  best  learning  algorithms  are  grounded  in  statistical  theory 
similar  to  the  one  underlying  the  current  approach.  We  conjecture  that  a  better  understanding  of  how  to 
automatically  acquire  probabilistic  models  and  behaviors  over  the  lifetime  of  a  robot  has  the  potential  to 
lead  to  new,  innovative  solutions  to  a  range  of  hard  open  problems  in  robotics. 

•  High-Level  Reasoning  and  Programming.  Current  research  on  probabilistic  robotics  predominately 
focuses  on  low-level  perception  and  control.  However,  the  issues  raised  in  this  paper  transcend  to  all 
levels  of  reasoning  and  decision  making  [5].  The  issue  of  probabilistic  high-level  reasoning  and  program¬ 
ming  for  robots  remains  poorly  explored.  Research  is  needed  on  algorithms  that  integrate  probabilistic 
representations  into  high-level  robot  control  (see  e.g.,  [34,  38, 69]). 

•  Theory.  The  groundedness  in  statistical  theory  makes  probabilistic  approaches  to  robotics  well-suited 
for  theoretical  investigation.  However,  existing  models  are  often  too  restrictive  to  characterize  robot 
interaction  in  complex  environments.  For  example,  little  is  known  about  the  consequences  of  violating 
the  Markov  assumption  that  underlies  much  of  today’s  work  on  localization  and  mapping,  even  though 
virtually  all  interesting  environments  violate  this  assumption.  Little  is  also  known  about  the  effect  of 


approximate  representation  on  the  performance  of  robotic  controllers.  A  better  theoretical  understanding 
of  probabilistic  robotic  algorithms  is  clearly  desirable  and  would  further  our  understanding  of  the  benefits 
and  limitations  of  this  approach. 

•  Innovative  Applications.  Finally,  we  see  a  tremendous  opportunity  to  apply  probabilistic  algorithms  to  a 
range  of  important  robotic  problems,  including  multi-robot  coordination,  sensor-based  manipulation,  and 
human  robot  interaction. 

We  hope  that  this  article  motivated  the  probabilistic  approach  to  robotics,  and  stimulates  new  thinking  in 
this  exciting  area.  Ultimately,  we  believe  that  probabilistic  algorithms  is  essential  for  a  much  broader  class 
of  embedded  systems  equipped  with  sensors  and  actuators. 
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